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Abstract —We study a lossy source coding problem with secrecy 
constraints in which a remote information source should be 
transmitted to a single destination via multiple agents in the 
presence of a passive eavesdropper. The agents observe noisy 
versions of the source and independently encode and transmit 
their observations to the destination via noiseless rate-limited 
links. The destination should estimate the remote source based 
on the information received from the agents within a certain 
mean distortion threshold. The eavesdropper, with access to side 
information correlated to the source, is able to listen in on one of 
the links from the agents to the destination in order to obtain as 
much information as possible about the source. This problem can 
be viewed as the so-called CEO problem with additional secrecy 
constraints. We establish inner and outer bounds on the rate- 
distortion-equivocation region of this problem. We also obtain the 
region in special cases where the bounds are tight. Furthermore, 
we study the quadratic Gaussian case and provide the optimal 
rate-distortion-equivocation region when the eavesdropper has 
no side information and an achievable region for a more general 
setup with side information at the eavesdropper. 

Index Terms —CEO problem, multiterminal source coding, 
secrecy constraints, eavesdropping, equivocation. 


I. Introduction 

As networks are becoming more distributed, their vulner¬ 
ability to malicious activities increases which in turn raises 
the concern on the security of such networks. Consequently, 
information-theoretic security as a concrete framework for 
analyzing secrecy in networks has gained attention among 
researchers 0. Information-theoretic security, which was 
initially introduced by Shannon 0, exploits different statis¬ 
tical characteristics of received information at the legitimate 
receiver and at the eavesdropper. Moreover, it makes no 
assumptions on the computational power of the eavesdropper, 
unlike the traditional cryptographic approaches for secrecy. 
Later, Wyner introduced the Wiretap channel model in [51 and 
showed that perfectly secure communication without a shared 
secret key is possible if the channel from the transmitter to 
the eavesdropper is a degraded version of the channel to the 
legitimate receiver. This result was generalized to broadcast 
channels with confidential messages by Csiszar and Korner in 
|6j. Subsequently, many extensions to this problem have been 
developed and studied in the literature (see, for instance, [[2), 
0, and references therein). 
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In this paper, we consider secrecy in a multiterminal source 
coding problem. In particular, we study the problem of convey¬ 
ing an information source to a single destination via multiple 
agents (encoders) in the presence of a passive eavesdropper. 
The agents have access to noisy observations of the source and 
are connected to the destination via noiseless rate-limited links. 
They do not cooperate or communicate to one another and are 
not required to estimate the source themselves. This scenario 
is of interest for many applications such as sensor networks 
or smart grid systems where reconstruction of the source at 
sensors and smart meters is not necessary. The distributed 
nature of such networks makes them more susceptible to 
eavesdropping. At each instant, the eavesdropper listens in on 
one of the links from the agents to the destination in order to 
obtain information about the source. In addition, it has access 
to side information correlated to the source. Since the link 
that will be compromised by the eavesdropper is unknown 
to the agents prior to their transmissions, each agent should 
protect its link in order to leak as little information as possible 
about the source. Our objective is to characterize the trade-off 
among agents’ transmission rates, incurred distortion at the 
destination, and the amount of information revealed to the 
eavesdropper. This setup can be viewed as the extension of 
the so-called CEO problem |7] in which secrecy constraints 
are considered. 

A. Related Work 

The chief executive/estimation officer (CEO) problem was 
motivated in 0 by a communication and distributed pro¬ 
cessing system analogous to a scenario in which a firm’s 
CEO is interested in information of a source that cannot be 
observed directly. The CEO assigns a group of agents to 
independently observe a corrupted version of the source and 
communicate their observations. The lossless variant of this 
setup was initially studied by Gel’fand and Pinsker 0. It was 
extended by Yamamoto and Itoh 0 as well as Flynn and 
Gray © to the lossy case with only two encoders for which 
an achievable rate-distortion region was derived. The model 
was generalized to the CEO problem with many encoders by 
Berger and Viswanathan |7j in which the trade-off between the 
end-to-end average distortion and sum of the rates at which 
the agents transmit to the CEO was studied. Multiterminal 
lossy source coding problems, including the CEO problem, 
are still open in general. However, for the special case of the 
quadratic Gaussian CEO problem fTT) , the sum-rate-distortion 
function for infinite number of agents with identical signal-to- 
noise ratios (SNRs) was derived by Oohama 1 12], and later, the 
complete rate-distortion region with arbitrary number of agents 
and SNR values was characterized by Prabhakaran et al. 
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and Oohama IB- More recently, Courtade and Weissman [ i5J 
gave the rate-distortion region of the CEO problem under the 
logarithmic-loss distortion measure. 

Secure lossless source coding with uncoded side informa¬ 
tion at the legitimate decoder and the eavesdropper was studied 
by Prabhakaran and Ramchandran IB with the assumption of 
no rate constraint on the encoder-decoder link. The minimum 
leakage rate was derived and it was shown that due to 
the side information at the eavesdropper, the usual Slepian- 
Wolf scheme IB is not always optimal. Lossless source 
coding with coded side information at the decoder (the so- 
called one-helper problem) and no side information at the 
eavesdropper was studied by Tandon et al. m where the 
rate-equivocation region was characterized. This setup was ex¬ 
tended by Giindiiz et al. IB with additional side information 
at the eavesdropper in which inner and outer bounds on the 
compression-equivocation rate region were derived that did 
not match in general. Secure distributed lossless compression 
of two correlated sources, in which both sources were to be 
estimated at the decoder, was considered by Luh and Kundur 
|20) without side information at the eavesdropper and by 
Giindiiz et al. IB with side information at the eavesdropper. 
These models were generalized by Salimi et al. |22) to the case 
where both the legitimate receiver and the eavesdropper have 
access to correlated side information and the eavesdropper 
can choose to intercept either links from the encoders to the 
decoder at each instant. In |22), inner and outer bounds for the 
compression-equivocation region were provided which were 
proved to be tight for several special cases. 

The extension to the lossy case was considered in |23)-[[25), 
and more recently by Villard and Piantanida |[26) in which 
inner and outer bounds on the rate-distortion-equivocation 
region were derived. The optimal characterization of the rate- 
distortion-equivocation region was first found in (24) for the 
lossy case with uncoded side information. Later in |26| , the 
optimal characterization for the lossless case was also derived. 
A different setup was considered by Kittichokechai et al. [271 
in which the eavesdropper can only access the coded side 
information, and the complete region was characterized under 
the logarithmic-loss distortion IB- Chia and Kittichokechai 
[ 28| studied the case when the encoder has access to the side 
information of the decoder. Tandon et al. [29) considered a 
scenario with two legitimate receivers and investigated the 
privacy of side information at one receiver with respect to 
the other one. An alternative approach to provide secrecy in 
source coding problems is based on having a shared secret key 
between the transmitter and the legitimate receiver {30) 
although we do not exploit this approach in our work. 


B. Contributions 

Our setup in this paper has two main distinctions from 
the aforementioned scenarios; first, the destination (CEO) is 
interested in estimation of the original source rather than 
the agents’ observations as in all prior works. Similarly, the 
secrecy constraints in our problem are on the equivocation of 
the eavesdropper with respect to the remote source, not to the 
observations of the agents. In fact, our setup is a generalization 



Fig. 1. The CEO problem with secrecy constraints. 


of the previous cases considered for lossy secure source coding 
problems. We extend our previous work {33) for the lossless 
variant of this problem to the lossy case and derive inner and 
outer bounds on the rate-distortion-equivocation region of the 
CEO problem with secrecy constraints. We also investigate 
the region in special cases where the bounds are tight and we 
show that for these special cases our results coincide with the 
previous results in the literature. 

In addition, we consider the quadratic Gaussian CEO 
problem with secrecy constraints and provide the optimal 
characterization of the rate-distortion-equivocation region for 
the case when the eavesdropper has no side information and 
an achievable region for a more general setup with side 
information at the eavesdropper. 


C. Notations and Organization 

In this paper, we use capital letters to indicate a random 
variable, small letters to indicate realization of a random 
variable, calligraphic letters to denote a set, e.g., X , and \X\ to 
indicate the cardinality of the set. The notation X n denotes the 
sequence {X-[. ..., X n }. The notion X — Y — Z shows that X, 
Y , and Z form a Markov chain, i.e., p(x , y , z) = p(x, y)p(z\y) 
or p{x,y,z ) = p(x\y)p(y,z). We define 1 M '■= 
for M £ N, and [x] + := max{0,x} for x £ R. Finally, 
1 k> 0 (x) : X —> {0,1} denotes the indicator function such 
that 1 r >0 (:t) = 1 for x £ R>o, and 1 r >0 (:t) = 0 otherwise. 

The rest of the paper is organized as follows: In Section [II] 
we describe the problem along with some definitions. Main 
results for inner and outer bounds on the rate-distortion- 
equivocation region are presented in Section III Then, we 
study some special cases of our results in Section IV where 
the region is completely characterized. The rate-distortion- 
equivocation region for the quadratic Gaussian case is given 
in Section [V] Finally, the paper is concluded in Section VI 


II. Problem Setting 

We consider the CEO problem with secrecy constraints 
as depicted in Fig. [T] In this setup, two non-cooperative 
and independent agents have access to length-n observations 
Y" and F 2 n , respectively, which are noisy versions of the 
source sequence X n . These observations are conditionally 
independent given X n . Each agent independently transmits 
a compressed version of its observation to the CEO over 
a rate-limited noiseless link. The CEO estimates the source 
sequence X based on the received information from the two 
agents. An eavesdropper, referred to as Eve, with access to 
side information E n correlated to the source sequence X n can 
eavesdrop only one of the links from the agents to the CEO at 
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each time instance to obtain as much information as possible 
about the source. Therefore, agents’ transmission rates should 
be such that the CEO can reconstruct the source reliably within 
a certain mean distortion threshold while simultaneously the 
equivocation at Eve is maximized. Eve’s equivocation, with 
respect to either links, corresponds to her uncertainty about 
the original source when she combines her side information 
with the information obtained from the link. We assume that 
Eve cannot access both links simultaneously as the links are 
noise-free and in such case she would be more powerful 
than the CEO for estimating the source due to her additional 
side information. The sequences X n , Yf\ Y 2 , and E n are 
independent and identically distributed (i.i.d.) according to 
joint distribution p(x,y 1 ,y 2 ,e) = p{x)p{y 1 \x)p{y 2 \x)p{e\x) 
over the finite alphabet X x J4 x y 2 x E. 

Let d : X x X —> [0, d max ] be a finite distortion measure. 
We define the component-wise mean distortion between two 
sequences x n , x n in X n as 

1 " 

d^\x n ,x n ):=-y j d{x i ,x i ). (1) 

»=l 

Definition 1: A (Mi, M 2 , n) -code for compression and 
transmission of the source by the agents with secrecy con¬ 
straints consists of an encoding function at each agent, f :/ : 
y™ —> Imj for j £ {1)2}, and a decoding function at the 
CEO, g : Im, x Im 2 X n . The equivocation rates for this 
code are defined as (X n \fj(Y J n ), E n ) for j £ {1,2}. 

Definition 2: A tuple (R\, f? 2 , Ai, A 2 , D) is said to be 
achievable if Ve > 0 there exists No £ N>o such that for all 
n > No there exists a sequence of (Mi, M 2 , n)-codes with 


E 


log (Mj) < n(Rj + e), Vj £ {1,2}, 
H{X n \fj{Y?),E n ) > n(Aj - e), Vj £ {1,2}, 
< D + e. 


S n \X n ,g(f 1 (Y 1 n )J 2 (Y 2 n ))) 


Let 1Z denote the rate-distortion-equivocation region defined 
as the set of all achievable tuples (Ri, R 2l Ai, A 2 , D). 


III. Inner and Outer Bounds on the 
Rate-Distortion-Equivocation Region 

A. Inner Bound 

Theorem 1: Let lZi n denote the region defined as the closure 
of the convex hull of the set of all tuples (i?i, f? 2 , Ai, A 2 , D) 
such that there exist random variables V \, V 2 , U\, and 
U 2 on some finite sets Vi, V 2 , U\ , and U 2 , respectively, 
according to the distribution p(x,yi,y 2 ,e,vi,v 2 ,u\,u 2 ) = 
p(x)p(y 1 \x)p(y 2 \x)p{e\x)p(u 1 \y 1 )p(u 2 \y 2 )p(v 1 \u 1 )p(v 2 [u 2 ) 
and a function X : IA\ x U 2 —> X that satisfy 


Ri>I{Up,Yi\U 2 ), 

(2) 

R 2 >I(U 2 -,Y 2 \U 1 ), 

(3) 

R 1 + R 2 >I(U 1 ,U 2 -,Y 1 ,Y 2 ), 

(4) 

Ai < [H(X\V U E) - 


+ I(U 1 -,Y 1 \V 1 ,X)] + , 

(5) 

A 2 < [H{X\V 2 ,E) - I{U 2 \Y 2 \V 2 ,Ux) 


+ I(U 2 -Y 2 \V 2i X )} + , 

(6) 


Ai + A 2 < [H(X\V u E) + H{X\V 2 ,E) 

- Wh,U 2 \Y u Y 2 \Vi,V 2 ) 

+ I(U 1 -,Y 1 \V 1 ,X) + I(U 2 -,Y 2 \V 2 ,X)] + , 0) 
Ai-f? 2 < [ff(A|yi,£;)-/([/ 2 ;y 2 |f/i) 


- IiU^Y^V!) + I(U 1 -,Y 1 \V 1 ,X)] + , 
A2-Ri< [H(X\V 2 ,E)-I(U 1 -,Y 1 \U 2 ) 


( 8 ) 


- I(U 2 ;Y 2 \V 2 ) + I(U 2 ;Y 2 \V 2 ,X)] + , (9) 

D>E\d(X,X(lh,U 2 ))]. (10) 


Then, we have lZi n C R. 

Proof: The proof is given in Appendix [A] ■ 

Proposition 1: In Theorem [l] it suffices to consider aux¬ 
iliary random variables Vj and Uj for j £ {1,2} with 
cardinalities \Vj\ < \yfi +9 and | Ufi < (l^j +9)(|3{j| +5), 
respectively (see Appendix [B| for the proof). 

The achievability scheme resulting in the inner bound is 
based on superposition coding and random binning at the 
agents, and joint decoding at the CEO. In particular, agent 
j first transmits the bin index related to the auxiliary random 
variable Vj with distribution piyfiuj) via the noiseless link. 
Then, the agents send the remaining information which is 
required for the CEO to be able to reconstruct the source based 
on the Wyner-Ziv scheme |34) . The detailed proof is given 
in Appendix [Aj however, we provide some intuitions on the 
results. Inequalities |2}-(|4]) and (jTOji are similar to the Berger- 
Tung bounds [35J, |36) that establish perfect estimation of U\ 
and U 2 at the CEO from which X can be reconstructed within 
the distortion limit D. In the equivocation bounds © and 
the first term corresponds to Eve’s uncertainty about the 
source after decoding the codeword vf based on the received 
bin index combined with her side information and the second 
term is the reduction in her uncertainty when receiving the 
remaining information transmitted to the CEO by the agents. 
Finally, the last term in (|5]i and stems from the fact that 
in contrast to previous works, the secrecy constraints are on 
Eve’s equivocation with respect to the original source while 
the transmitted information by the agents are functions of their 
respective observations and not the source, resulting in an 
increase in Eve’s uncertainty. Inequalities ([H} and (|9} depict a 
trade-off between Eve’s equivocation and transmission rates, 
implying that each link’s transmission rate limits the other 
link’s equivocation rate. 

Remark 1: The region of Theorem [T] can also be obtained 
by constructing six different codes achieving the corner points 
shown in Tables M and using the time-sharing technique 
between these points. Each corner point is achieved using a 
four-step communication to transmit variables V t , V 2 , U\ , and 
U 2 to the CEO with different decoding orders, provided that V :J 
is decoded prior to Uj for j £ {1,2}. In each step, previously 
received and decoded information at the CEO is used as side 
information for the current decoding step. Each code employs 
superposition coding, with Vj as the first layer and Uj as the 
second layer, and random binning based on the available side 
information at the CEO in each communication step. 
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TABLE I 

Corner points of the inner region corresponding to different decoding orders: rates and distortion. 


Comer point 

Decoding order 

Ri 

r 2 

D 

1 

V 2 ,U 2 ,Vi,Ui 

7(Ei;Yi|E 2 ) 

7(E 2 ;Y 2 ) 

m[d(x,x(uuu 2 ))] 

2 

v 2 ,v 1 ,u 2 ,u 1 

7(Vi;Yi|V 2 ) + /(Ei;Yi|Vi,E 2 ) 

7(Y 2 ;Y 2 ) + 7(E 2 ;Y 2 |Vi,V 2 ) 

E[d(X,.Y(Ei,E 2 ))] 

3 

Vi,V 2 ,U 2 ,Ui 

7(Vi;Yi) + /(Ei;Y!|Vi,E 2 ) 

7(E 2 ; Y 2 |Vi) 

E[d(X,.Y(Ei,E 2 ))] 

4 

v 1 ,u 1 ,v 2 ,u 2 

7(Ei;Vi) 

7(E 2 ;Y 2 |Ei) 

E[d(X,X(Ei,E 2 ))] 

5 

Vi, V 2 ,Ei,E 2 

7(Vi;Yi) + /(Ei;Yi|Vi,V 2 ) 

I(V 2 ;Y 2 \Vi) + I(U 2 -,Y 2 \V 2 ,Ui) 

E[d(X,X(Ui,U 2 ))} 

6 

V 2 , Vi , Ei , U 2 

7(Ei;Yi|Y 2 ) 

/(V 2 ;Y 2 ) + /(E 2 ;Y 2 |\/ 2 ,Ei) 

E[d(X,X(Ei,E 2 ))] 


TABLE II 

Corner points of the inner region corresponding to different decoding orders: equivocation rates. 


Comer point Decoding order 

1 v 2 ,u 2 ,v 1 ,u 1 

2 V 2 , Vi, U 2 , Ui 

3 V 1 ,V 2 ,U 2 ,U 1 

4 V 1 ,U 1 ,V 2 ,U 2 

5 Vi,V 2 ,U lt U 2 

6 V 2 ,Vi,Ei,E 2 


Ai 

[H(X\Vi ,E)-I(U 1 ;Y 1 \Vi,U 2 ) + J(Ei; Yi|Vi, X)] + 
[H(X\Vi,E) - 7(Ei; Yi|Vi, U 2 ) + 7((7 i; Yi| Vi, X)] + 
[tf(X|Vi,E) - 7((7i; Ti|Vi, U 2 ) + 7(I7i; Yi|Vi, X)] + 
[H(X\V !, E) - I(Ui ; Yi | Vi) + J(Ei; Vi | Vi, X)} + 
[H(X\Vi , E) - I(Ui ; Yi | Vi, y 2 ) + /((7i; Vi | Vi, X)] + 
[H (A'| Vi, E) - J((7i; Yi | Vi, V 2 ) + /(Ei; li | Vi, X)] + 


^2 

[H(X|Y 2 ,E) - I{Ur,Y 2 \V 2 ) + I(U 2 -,Y 2 \V 2 ,X)\ + 
[H(X IVi, E) - I(U 2 ; Y 2 1 Vi, Vi) + I{U 2 ; Y 2 1 V 2 , -Y)] + 
[E(X| y 2 , E) - 7(E 2 ; Y 2 1 Vi, V 2 ) + I(U 2 ; Y 2 1 V 2 , X)] + 
[H(X\V 2 ,E)-I(U 2 -,Y 2 \V 2 ,U 1 ) + I(U 2 -,Y 2 \V 2 ,X)] + 
[H(X\V 2 ,E) - I{U 2 -,Y 2 \V 2 ,Ui) + I(U 2 ;Y 2 \V 2 ,X)] + 
[H(X\V 2 , E) - I{U 2 ; Y 2 | Vi, Ei) + 7(E 2 ; Y 2 | V 2 , X)] + 


B. Outer Bound 

Theorem 2: Let lZ out denote the region defined as the 
closure of the set of all tuples (f?i, R 2 , At, A 2 , D) such that 
there exist random variables V\, V 2 , U -\, and U 2 on some finite 
sets Vi, V- 2 , U \, and U 2 , respectively, which form Markov 
chains Vj — Uj — Yj — ( X,E,Yji ) for j,j' £ {1,2} with 
j ^ j' , and a function X : U\ x U 2 —> X that satisfy 


The inner bound is obtained by setting Y\ = X, V 2 = U 2 , and 
removing the constraints on A 2 in Theorem [T] and the outer 
bound can be proved similar to the proof of Theorem [2] 
Corollary 2: In the described one-helper problem with se¬ 
crecy constraints, if R 2 > H{Y 2 ), the helper’s sequence can 
be reconstructed by the destination losslessly. Then, the rate- 
distortion-equivocation region is characterized by 


Ri >/(C7i;ri|t7 2 ), 

(11) 

R 2 >I(U 2 -Y 2 \U 1 ), 

(12) 

Ri+R 2 >I{U 1 ,U 2 ;Y 1 ,Y 2 ), 

03) 

Ar < H(X\E) - I(X; Vi\E), 

(14) 

A 2 < H(X\E) -I(X;V 2 \E), 

(15) 


Ar -R 2 < H{X\E) — I(X; Vi\E) — I(X; V 2 \Vi,E), (16) 


A 2 - Ri < H(X\E) ~ I(X-,V 2 \E) - IiX-V^E), (17) 


D > E 


d(X,X(U u U 2 )) 


(18) 


Then, we have 1 Z C 77, out . 

Proof: The proof is given in Appendix [C] ■ 

Proposition 2: In Theorem [2] it suffices to consider aux¬ 
iliary random variables V t and Uj for j £ {1,2} with 
cardinalities \Vj\ < |^-| + 7 and \Uj\ < (|^-| + 7)(|^| + 3), 
respectively (see Appendix |D| for the proof). 


Ri > I{X'Ui\Y 2 ), (19) 

Ai< [H(X\Vi,E) - I(Ui;Yi\Vi,Y 2 )] + , (20) 

A 2 < H(X\Y 2 ,E), (21) 

D>E\d(X,X(U u Y 2 ))], (22) 


where the auxiliary random variables V\ and U\ satisfy the 
Markov chain V\ — U\ — X — ( E , Y 2 ). 

The achievability proof follows from the proof of TheoremlT] 
by setting Y\ = A' and V 2 - U 2 = Y 2 . Inequalities 0-0 
are inactive for this setup. The converse proof is given in [26] 
Theorem 3] for the secure lossy source coding with uncoded 
side information. Note that if Eve intercepts the helper’s link, 
it can also reconstruct the helper’s sequence Y,J‘ losslessly. 

Corollary 3: For the lossless one-helper setting, i.e., D = 
0, if R 2 > H(Y 2 ), the rate-equivocation region writes as: 


IV. Special Case: The One-Helper Problem with 
Secrecy Constraints 

If Agent 1 has access to the source sequence X n , our setup 
reduces to the lossy source coding problem with a helper and 
an eavesdropper who can choose to listen in on either source- 
destination or helper-destination links. 

Corollary 1: In the above setting, if we additionally assume 
the helper’s link is perfectly secure, our results coincide with 


the results given by Villard and Piantanida 126 Theorem 3]. 


Ri > H[X\Y 2 ), 

(23) 

AjKIiX-Y^-IiX-E^), 

(24) 

A 2 < H(X\Y 2 E). 

(25) 


The achievability proof follows from the proof of Theorem |T| 
by setting U\ = Y\ = X and V 2 = U 2 = Y 2 . The converse 
proof is similar to the proof given in [16]. 

Corollary 4: For the lossless one-helper setting, i.e., D = 
0, if the eavesdropper has no side information, the rate- 
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Fig. 2. The quadratic Gaussian case with no side information at Eve. 

equivocation region is characterized by 

Ri > H(X\U 2 ), (26) 

R2>I(Y 2 -U 2 ), (27) 

Ar < I(X; U 2 ), (28) 

A 2 < H(X\U 2 ). (29) 

The achievability is a special case of Theorem |T| and obtained 
by setting V\ and E to be constants, U-\ = Y\ = X, and 
V 2 = U 2 . The proof of converse is given in Appendix [E] 





quadratic Gaussian case with no side information at Eve and different 
distortion constraints. 


V. The Quadratic Gaussian Case 


In this section, we study the Gaussian CEO problem with 
secrecy constraints and quadratic distortion measure. 

Let X be a Gaussian source, i.e., X ~ Af(0,a\). The 
observations at the agents are modeled as Y 3 = X + Nj for 
j £ {1,2}, with Nj ~ A/"(0, crjy.), where Gaussian random 
variables X , Ni, and N 2 are mutually independent. 

First, we consider the case where the eavesdropper has 
no side information. The model is depicted in Fig. [2] and 
the following theorem provides the complete rate-distortion- 
equivocation region for this Gaussian setup. 

Theorem 3: In the quadratic Gaussian CEO problem with 
secrecy constraints, the rate-distortion-equivocation region is 
characterized by the set of all tuples (f? 1; R 2 , Ai, A 2 , D) 
satisfying 


Ri 

R-2 

R\ + f?2 
A! 

a 2 


Ai + A 2 
Ai — R 2 
A 2 — R\ 


1 , 1 1 , ( 1 1 - 2~ 2r2 

^■ + 2 1 »*D-2 1 »*U + ^rr 

1 1 


> ^-2 + 71 log---log —n~ + 


D 2 ° V (7 


1 - 2 


—2n 


X 


o 


Ni 


> r\ + r 2 + 


7 log i;-7 los 5' 


< g log( 27 recri-) - - log — 


1 


+ o lo S 32- 


1 


1 - 2 -2t ' 2 


'n 2 


< i log( 27 recr|-) - \ log 


1 


1 




a N 1 

1 

1 

1 

D 

_ 2 

l0g ^ 


+ 2 l0 S 1 

< log( 27 re) - i 

< ^ log( 27 re) - ^log-^ -r 2 , 

< ^ log( 27 re) - ^iog-^ -G, 


1 - 2~ 2ri 


, (30) 

, (31) 

(32) 

(33) 

(34) 

(35) 

(36) 

(37) 


for some (rr, r 2 ) € ®>o that satisfy 


1 1 
D~a% 


1 - 2“ 2ri 1 - 2 _2r2 
H- o -1 - ■ 


(38) 


J N 1 u N 2 

Proof: The proof is given in AppendixlF] ■ 

An example of the region of Theorem Bps illustrated in 
Fig. [3] for different distortion constraints. 

Remark 2: We note that equivocation as the secrecy mea¬ 
sure in the finite alphabet setting represents the uncertainty 
of Eve about the source, but in the Gaussian setting, this 
interpretation is not quite valid. However, based on 137 The¬ 
orem 8.6.6], we can relate the equivocation rate (normalized 
differential entropy) to the estimation error at the eavesdropper. 
That is, we define the secrecy measure in the Gaussian setting 
as 


-E 

n 


| X n - Z {n) {fj{Y^),E n ) | 


>—(39) 


for j = 1,2, where (fj(YJ l ). E n ) is an estimator of A'” 
at the eavesdropper. Then, we have A j = 1 log(27r edj), and 
in this sense, equivocation rate provides a lower bound on the 
normalized distortion at Eve. Moreover, we can relate this to 
the information leakage rate as 


~l(X n -, fj(Yp,E n )) = h(X) - U(fj(Yp,E n )) 

< HX) - A j 

= H x ) - \ log(2?r edj), (40) 

which is also in line with the result in (38i Theorem 7.3]. 

Next, we consider the case where Eve has access to addi¬ 
tional side information correlated to the source as shown in 
Fig. 0 We model this side information as E = X + Ne where 
Ne is a Gaussian random variable with Ne ~ Af(0, cr% E ) and 
is independent of A, Ni, and N 2 . The following theorem gives 
an inner bound for the rate-distortion-equivocation region of 
the quadratic Gaussian CEO problem with secrecy constraints 
and side information at the eavesdropper. 
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Theorem 4: In the quadratic Gaussian CEO problem with 
secrecy constraints and side information at the eavesdropper, 
a tuple (i?!, f? 2 , A 1; A 2 , D) is achievable if 

1 11 / 1 1 — 2~ 2r 
Ri > n + - log — - - log ( —-f- - 


1 


D 2 
1 1 


'x 


'n 2 


i ?2 > r 2 + -log—--log —n~ + 


D 2 


1 1-2 


— 2n 




a 


tv. 


Ri + R 2 > n + r 2 + ^ log ^ log 

X 


1 


Ai < - log(27re) - - log — 


1 


1 


ai ' a 2 


1 . 1 1, 
-2 l0g D + 2 l0g 


1 

J x 


X u N e 
1 - 2" 2r2 ^ 


1 


1 


A 2 < x log(27re) - x log - 5 - 


'n 2 


'AT 


1 


1 1 


- Ttlog-R + ttlog ~T 


2 0 D ' 2 

Ai + A 2 < log(27re) — log 


1 1 - 2" 2ri 


'x 

1 

J x 

\ lo S 4- + T 1 + T 2 

z cr x 


J N 1 


(41) 

(42) 

(43) 


+Ti, (44) 


+T 2 , (45) 


'at e 


'X 


; »1 


+ 2 l0g 


'I T 1-2- 2 ’-! 


«1 


1 — 2 - 


“n 2 

1_2-2t- 2 


- 1 log — 
2 B D 


1r >0 (^1+^2), 


A , - R -2 < - log(27re) - x log — 


'IV E 


1, 1 

— 2 ° g ~D~ r2 


T, 


+ i og 


; "i 


T 1 _ 2 -2r 1 


«1 


1 - 2 - 


A 2 - R X < ^ log(27re) - * log 


at 2 

1 —2 -2r 2 

1 




1 

T N e 


1, 1 

- 2 l0g o- ri 


To 


+ ^ log 


a 




JfT !_ 2 - 2 r! 


where l* 


\ 1 _ 2 -2i-i ~ l _ 2 -- r 2 / 

j(-) is the indicator function and 


(46) 


(47) 


1 r>0 (T 2 ), (48) 


Ti := max l 0, ^ log |1 + 


_ -2 

1—2 -2r 2 u Ne 


N 1 I 

l_ 2 -2r 1 + <7 


T 2 := max j 0, ^ log 11 + 




«2 


+ a 


V \ l _2~ 2 r 2 

for some (n, r 2 ) eK 2 >0 that satisfy 

1 - 2“ 2r2 



1 

D - - 2 


1 1 - 2" 2ri 
H-o- 


a 


A' 


'JVi 


j n 2 


Proof: The proof is given in Appendix [G] 


(49) 


(50) 


Ni 



Ne 


Fig. 4. The quadratic Gaussian case with side information at Eve. 


Note that if there is no correlation between Eve’s side 
information and the source, i.e., o% E —► 00 , the region of 
Theorem [4] coincides with the one in Theorem 0 


VI. Conclusion 

We studied the extension of the CEO problem with secrecy 
constraints. This setup is of interest to communication scenar¬ 
ios such as sensor networks or smart power grids in which 
links are vulnerable to eavesdropping. We derived inner and 
outer bounds on the rate-distortion-equivocation region in the 
discrete case. We also showed that the results that were derived 
for the one-helper problem with secrecy constraints in p8| 
and 1261 can be obtained as special cases of our results for the 
CEO problem with secrecy constraints. In addition, we provide 
the optimal region for the quadratic Gaussian case when Eve 
has no side information as well as an achievable region for a 
more general case. In this work, we have considered noise- 
free links from the agents to the CEO, however, it would 
be interesting to investigate the effects of noisy channels in 
this problem. Moreover, extending this problem to include 
more agents and eavesdroppers with possibly different side 
information is another direction worthwhile investigating. 


Appendix A 

Proof of TheoremQ] The Inner Bound 
We first state the following lemma that we use in the proof 
of Theorem[I] The lemma follows from j2] Section 2.3], 
Lemma 1: Let C n be a random codebook and s be the 
corresponding codeword index of V n from this codebook. 
Let Pr ^(V n (s), E n ) £ 7^"^ —> 1 as n —> 00 , where 
denotes the set of jointly e-typical n-sequences. Then, we have 

H(E n \ S ,C n )<n(H(E\V) + e). (52) 

Proof of Lemma [7} Let Z be a binary variable such that 

z = | 1 if {V n ,E n ) £ Te [n) 

\ 0 otherwise. 

If Pr [Z = 0) < e for sufficiently large n, we have 

H(E n \s,C n ) < H(E n \V n ,C n ) 

< H(E n \V n ,Z) + H{Z) 

< Pr (Z = l)H(E n \V n , Z=l) 

+ Pr (Z = 0)H(E n \V n , Z = 0) + H(Z) 
<H(E n \V n ,Z=l)+nelog\£\ + H(Z) 

< Y P(v n \Z = l)H(E n \V n =v n ,Z= 1) 

v ’ ■er t (n) 


(51) 
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+ ne log \£\ + H(Z) 

< Y, p(v n \Z=l)log\T t M (E\v n )\ 

V ”£% M 

+ ne log \£\ + H 2 {e) 

<n(H(E\V) + e'), (53) 


where 11 2 { ) is the binary entropy function. The last inequality 
is due to properties of jointly typical sequences 139 Chap¬ 
ter 2 ], ■ 


Now, we proceed to prove Theorem |T| 

Let Vj, V 2 , U\, and U 2 be random variables on some 
finite sets Vi, V 2 , U\, and U 2 according to the joint distribu¬ 
tion p{x,yi,y 2 ,e,v 1 ,v 2 ,u 1 ,u 2 ) = p{x)p{y 1 \x)p{y 2 \x)p{e\x) 
p{ui\yi)p(u 2 \y 2 )p{vi\ui)p(v 2 \u 2 ), along with a function X : 
U\ x U 2 —> X satisfying the conditions of Theorem [T] 

Codebook generation-. For fixed conditional distribu¬ 
tions p(uj \ y,j) and p(v.j\uj), j = 1 , 2 , randomly gen¬ 

erate 2 "(FL/T j)+ £i ) independent codewords t)"(sf) of 
length n according to nr= = 1 Pvj(vj,i(sj)), where sj G 
{1,..., 2 n P( v l ; Tj)+ e i)j. Then, divide them into 2 n< ’ Rv j +(2 ' > 
equal-sized bins, indexed by bj G {1,... ,2 n ^ Rv i +e2 ^} 
and denoted by {Bj(bj)}. For each codeword Vj’(sj), 
randomly generate independent sequences 

Uj(sj, s') according to n"=i p u, ( u j,i( s j > s j))> and divide 
them into 2 n(Ru :i +fi> equal-sized bins, indexed by Wj G 
{1,..., 2 n< ' Ru j +e4 ' > } and denoted by {Bj(sj,Wj)}. Define 
Rj = Ii\ j + R.jjj for j = 1,2. The codebook is revealed 
to the agents, CEO, and Eve. 


Encoding: Assume that the sequence y" is observed by 
Agent j, j = 1,2. Find a codeword Vj-(sj) jointly typical 
with y ". If there is more than one such codeword, select one 
uniformly at random. If there is no such t>”, select one out 
of 2 ”( / (Ff;Tj)+ e i) uniformly at random. Given Vj-(sj), find a 
codeword uj(sj,s'j) jointly typical with y”. If there is more 
than one such codeword, select one uniformly at random. 
If there is no such u", select one out of 2 ni ' Ii ' U P’ Y : j\v ;j )+f. 3 ) 
uniformly at random. The agent transmits the bin indices bj 
and Wj of the codewords Vj-(sj) G Bj{bj) and u"(sj,s') G 
Bj(sj,Wj), respectively, i.e., fj(yj) = ( bj,wj). 

Decoding at the CEO’. Given the received messages from 
both agents, J\ = ( 6 i,uq) and J 2 = ( b 2 ,w 2 ), find a 
unique index tuple (s 1; sj, s 2 , 4) such that the codewords 
(u”(si), «”(si, sj), v 2 (§ 2 ), u 2 {s 2 , 4)) are jointly typical, and 
they are in the bin indexed by ( 61 , wi, b 2 ,w 2 ). If there is such a 
unique index tuple, compute the source estimate component¬ 
wise as Xi = gi(Ji,J 2 ) := X(u lti {s 1 ,s' 1 ),U 2 ,i{s 2 ,s 2 )) for 
i = 1 ,..., n; otherwise set the output to an arbitrary sequence 
in X n . 


Error analysis: Let (si, sj,S 2 ,sj) and (si, sj, 4, 4) be 

the chosen indices at the encoders and decoder, respectively. 
Let Pr(E) denote the probability of an error event during 
encoding and decoding steps. We now show that this prob¬ 
ability, averaged over all possible codebooks, tends to zero as 
n — > 00 provided that conditions of Theorem [T] is satisfied. 
Consider the following error events in the encoding steps (for 


j,f = 1,2, and j 4 j'): 


En = 


{(S", Y?, Y 2 n , E n ) i T} n) {S, Y^E)}, 

Ei = Eg n { (ypiSj), YJ 1 ) i T} n \Vj,Yj)}, 

Wsj G {l,...,2 n ^^d+«i)} ; 

E 2 = Eg n Ej n {([f"(sj, s'-), 17) i T s {n) {Uj,Yj\v^{ Sj ))} : 

Vsj G {!,...,2 n ( J (^-; y ilV,0+«3)} ) 


E,= 


E, = 


E,= 


n E i)n{{v?( aj ),Y?,Y?) i r/ n) (Ff,yi,y 2 )}, 

t =0 

(fl E ?)n{(t/”(sj,s'),Yj n ,y 2 n ) 


t =0 


$‘Ts n Hu j ,Y u Y 2 \v?(a j 


Er = 


( D E 0 n {(^^),^(sT,s'O,^,^) 

£—0 

tr^iVj^j.^Y^isj,))], 

( f] Eg) n{ (Vj"(st), vp(s 2 ), Y™, Y 2 n ) 

t—0 

^r 5 (n) (Pi ,v 2 ,y u y 2 )}, 

E 7 =( H E*) n {(C/” ( Sl , sj), U 2 (s 2 , sj),!"", Y 2 ) 

t—0 

i rj n) {U 1 ,U 2 ,Y 1 ,Y 2 \vi{si),v 2 {s 2 ))Y 

Next, consider the following error event in decoding step: 

7 

e 8 = (n Eg) n{(vr(5 1 ), u?(Si, sj), v 2 n (s 2 ), U 2 (s 2 , 4)) 


t—0 


€T} n HVi,U 1 ,V 2 ,U 2 ), 
for some (si, sj, h, 4 ) 4 («i, 4 , s 2 , 4 ), 

s.t. (v?*(Si), u?(h, Sj), v 2 n (h), 4 )) 

G BM x Bj(sr,tttr) x B 2 (b 2 ) x B' 2 {s 2 ,w 2 )Y 

Finally, by the union of events bound, the probability of error 
in the encoding and decoding steps is upper bounded as 


S 0 

Pr(E)<Pr(ljE t ) <^Pr(E t ). 


(54) 


t =0 


t=0 


We proceed to bound each term in ( |54| . From properties of 
typical sequences, Pr(Eo) vanishes as n —> 00 . By covering 
lemma [39, Lemma 3.3], Pr(Ei) and Pr(E 2 ) tend to zero as 
n —> 00 . For j, j' = 1,2, and j 4 j', since {Yp\VJ l (sj) = 
vj, YJ 1 = yj 1 } - T\'- =1 p(yj\i\yj,i), by conditional typicality 
lemma [39 Chapter 2], Pr(E 3 ) tends to zero as n -t 00 . 
Similarly, as {Yp\V?{s 0 ) = v?, U?( Sj , s'.) = u^Y? = 
Vj} ~ nr=i p{yj\i\yj,i)’ P r ( E 4 ) also vanishes as n -> 00 . To 
bound Pr(E 5 ), let (i^.u^yJSy?) G T} n \Vj>, U j ',Y 1 ,Y 2 ). 
Then, Pr (VJ 1 (sj)| Vp(sj >) = t>£, U?(s Jt s') = u^Y?^ 


y™, YJ 1 = y 2 ) = piVjlyf), and by Markov lemma [39 




Lemma 12.1], Pr(Es) tends to zero as n —> oo. Using similar 
steps and based on Markov lemma, Pr(Eg) and Pr(E 7 ) also 
tend to zeros as n —7 oo. 

As can be seen from Eg, in the decoding step, an error oc¬ 
curs if the decoded codewords are jointly typical and they are 
in the bin indexed by ( 61 , w x , 62 , w 2 ), however, the decoded 
tuple (si, 4 , s 2 , 4) °f codeword indices are different from 
the chosen ones at the encoders, i.e., (si, 4, s 2 , sO- We split 
this event into eight possible events (other events result in the 
same constraints as one of these eight events) and bound its 
probability using the union of events bound as follows: 


Pr(Eg) = Pr((sr, s{ 

5 ^2 

,4 

) 4 (si, 

4 , s 2, S 2 

)) 

(55) 

< Pr(si 7C Si) 





(56) 

+ Pr(si = 

Si, 

S2 

4 S2) 



(57) 

+ Pr(si 7^ 

Si, 

S2 

= s 2 ) 



(58) 

+ Pr(sr = 

Si, 

s'i 

4 Sr,s 2 

= s 2 ) 


(59) 

+ Pr(si = 

Si, 

4 

(N 

<C0 

II 

4 S2) 


(60) 

+ Pr(si 7C 

Si, 

S2 

= s 2 ,4 

= s' 2 ) 


(61) 

+ Pr(si = 

Si, 

s'i 

= s{, s 2 

= s 2 , S2 

4 4 ) 

(62) 

+ Pr(si = 

Si, 

s'i 

4 Sr,s 2 

= S 2 ,S2 

= s' 2 ). 

(63) 


Now, we consider each of the terms in ([56])—(|63l), 


Next, we bound the probability of the event in which Si is 
correctly decoded but not s 2 , i.e., © as 

Pr(si = si,s 2 4 s 2 ) 

= Pr^3s 2 4 s 2 ,4>4, s.t. «(si),<(si,si), 
V^(S 2 ),U^(s 2 ,s' 2 )) G Tt\vuU U V 2 ,U 2 ) 
n {Si (hi) X Bi(*!,«!) X £2(62) X B' 2 (s 2 ,w 2 )}) 

< Pr((ur, Uj\ V 2 n , U 2 n ) G T s (n) (Vu U u V 2 ,U 2 )\ 

(00 U?) G Tj n) (V u U x ), (00 U?) € r} n \v 2 , u 2 j) 

. 2«U(V2;t'2)+ei-itv 2 +E J 2 = iP(^;UIU)+e3-fl t / J .]) 

< 2 n (I(y2iY2)+e\— flv 2 +52 2 =iP(^ , i'^2'l^i)+ e 3 — Ruj]~I(Ui;U 2 )+ri) 

( 66 ) 

Therefore, Pr(s 7 = Si, s 2 4 s 2 ) vanishes as n —> 00 if 
2 2 

Rv 2 + E * 1{ y* Y 2 )+E WiMM) - m-, u 2 ) 

i=1 3=1 

— I{U\\ 0 ( 0 ) + I(U 2 ; Y 2 \U\) 

= I(Ur, O, 0 | 0 ) + I(U 2 -, 0 , 0 ( 0 , U x ) 
I(U u U 2 \Y x ,Y 2 I Ur), (67) 


Pr(si 4 si) 

= Pr^3si ^ si,s , 1 ,s 2 l s , 2 , s.t. «(si), u"(si, 4 ), 

O(* 2 ), u n 2 (s 2 , 4)) e T s (n \vu U U V 2 , U 2 ) 
n {Si(60 x B[(h,w X ) X B 2 (b 2 ) X B' 2 (S 2 ,tu 2 )}) 

< Pr((00 OO 00 U?) G T S M (V u U u V 2 , U 2 ) I 

(00 ccd g r/ n) ( 0 , cco, (00 cc 2 o e r/ n) (o, cc 2 )) 

( < 2- n ( I ( v oUi;V 2 ,U 2 )-v) 

. 2 n 52j=i P(Ui>Ti)+ e i — +/(t7j;Tj|Vj)-l-e3— 

= 2 ra (57?=i ;U')+ei -Rv 3 . +i{u 3 -,Yj \ Vj )+z 3 -R Uj ]-UjJi-,u 2 )+,,) 

(64) 


where (a) is due to the mutual packing lemma 139 
Lemma 12.2]. Therefore, Pr(si 4 Si) vanishes as n -) 00 
if 


E R y : , + R u } > E 7 (4; O) + I(Uj\Yj 10) - I(f/ i; CC 2 ) 

i=i j'=i 

= I(Ur,Yi) + I(U 2 ; O) - /(CO; CO) 

— I(Ui,U 2 -, 0,0), (65) 


where (a) and ( 6 ) follow from the long Markov chain V\ — 

J7i - Pi -Y 2 -U 2 - V 2 . 


where (a), ( 6 ), and (c) follow from the Markov chain V\—Ui — 
O — Y 2 — U 2 — O- With similar steps, Pr(si 4 si, s 2 = s 2 ) 
in ( | 58 ] tends to zero as n —> 00 if 

2 

R Vl +J 2 R Uj> OCCi, C/ 2 ; 0,010). (68) 

j =1 

We now bound ([ 59 ] where si and s 2 are decoded correctly: 

Pr(si = Si, 4 4 a[, s 2 = s 2 ) 

= Pr(3si 4 si,4> s t - (<(si).<(siOi), 
v2(s 2 ),u%(s 2 ,s' 2 )) G Tg n \v x , Ui,V 2 ,U 2 ) 
n {Sr(60 x B[(si,wi) x S 2 (6 2 ) x S'(s 2 ,«> 2 )}) 

= Pr(((V7, ITT),00 U 2 ) G T 5 (ti) (Vi, C/r, O, C/ 2 ) 

n (00c/r ,00 G T} n) (Vi,u u v 2 ) 

n {Si (60 X Sl(si,iuO X B 2 (b 2 ) X S'(s 2 ,tU 2 )}) 

= Pr(((oo c/r), 00 c/ 2 0 g r/ n) (o, 0,0,^)! 

(00 CCD G r/ n) (o, cco, (00 CCD g r/ n) (o, cc 2 )) 

• Pr((O0 00 00 G r/ n) (0, D, 0)1 

(00 CCD G T s (n \ O, cco, (00 00 G T/ n) (0,0)) 

n {Si(60 X Sj(si,tuO X S 2 (6 2 ) x B' 2 (s 2 ,w 2 )}^ 

^ 2~n(I(Vi,Ui;U 2 \V 2 )-ri) . 2 -™(/(t/i;V 2 |Vi)- 77 ) 

. 2 n Y 2 j ^R U J’ Y j\V j )+e 3 -R Uj ] 
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= 2 n ^ = Anu j -,Y j \ v j)+^-Ru j ]-i(v 1 ,u 1 -,u 2 \v 2 )-nu 1 -,v 2 \v 1 )+ 2 r l ) 

(69)’ 

where (a) follows from the joint typicality lemma and the fact 
that Up ~ Yli=iP(u 2 ,i\v 2ti ), and Up ~ Yl?=iP( u h 
Therefore, Pr(si = Si , s\ 7 ^ s\ , s 2 = s 2 ) vanishes asn->oo 
if 

2 

^ f?^. > 7(£/i; FilVr) - J(C/i; y 2 |Pi) + /(tf 2 ; W) 
i=i 

-I(Vi,(7i;[/ 2 |y 2 ) 

= /(C/i; Y 1 \V 1 , V 2 ) + I(U 2 ; F 2 |Vi, J7i, Vj,) 

= /(J7i; n, r 2 1 Vi, V 2 ) + I{U 2 ; Y x , F 2 |ki, U x , P 2 ) 

= /((7i, U 2 ] Y\, F 2 |Vi, V 2 ), (70) 

where (a) is due the Markov chains U\ — Y\ — V 2 and U 2 — 
Yi - (C/i, Vi), ( 6 ) is due to L/,- - Yj - Y y . 

Next, for ( [60] ), we have 

Pr(si = si, si = si, s 2 7 ^ s 2 ) 

= Pr^3s 2 7 ^ s 2 ,s' 2 > S - L «(si),<(si,si), 

<(S2),^(S2,S'' 2 )) G Tj n \V u U u V 2 ,U 2 ) 

n {BM x B[( Sl , Wl ) x S 2 ( 6 2 ) x Bi(s 2 ,tu 2 )}) 

< Pr((^T, Up, V 2 n , U 2 n ) G 7f n) (V u U u V 2 , U 2 )\ 

(vp, UP) G r/ n) (Pi, up,, (VP, UP) G r s (n \v 2 , up) 

. 2 n ( I (V2-,Y 2 )+e 1 -Rv 2 +nU2-,Y 2 \V 2 )+e 3 -R U2 ) 

< 2 n ( / (^2T'2)+£i--Rv 2 +/(G 2 ;y 2 |t/ 2 )+£3--Rc/ 2 -/(t/i;tt 2 )+r ? )_ 

(71) 

Therefore, Pr(si = Si, si = si, s 2 7 ^ s 2 ) vanishes as n —>■ 00 

if 

Rv 2 + Ru 2 > I(V 2 ; Y 2 ) + I(U 2 ;Y 2 \V 2 ) - I(Jh-,U 2 ) 

= I(U 2 ; Y 2 1 UP, (72) 

which follows from the Markov chain V 2 — U 2 — Y 2 — U\. 
Similarly, Pr(si 7 ^ Si, s 2 = s 2 , s 2 = s 2 ) in ( | 6 T[ tends to zero 
as n —► 00 if 

i?Vi +-Rt7i > I(Ui',Yi\U 2 ). (73) 

We proceed to bound ( |62] > where (si,si,s 2 ) are decoded 
correctly: 

Pr(si = si, si = si, s 2 = s 2 , si 7 ^ s' 2 ) 

= Pr(3s ' 2 7 ^ s' 2 , s.t. «(si),<(si,si), 

v 2 (s 2 ), u%(s 2 , hats 2 )) G T^\Vi,U u V 2 ,U 2 ) 
n {Bi(&i) x Biisr.W!) X £ 2 ( 6 2 ) x Si(s 2 , W2 )}) 

< Pr((V7*, UP, V 2 n , U 2 n ) G T s (n) (V j, U 1} V 2 , U 2 )\ 
((vp,up),vp) er} n) (Vx,UuV 2 ), 
(vp,up)eT s {n] (V 2 ,U 2 )) 

. 2 n U(U 2 ;Y 2 \V 2 )+e 3 -R U2 ) 


< 2MI(U2;Y 2 \V 2 )+e 3 -R U2 -I(V u U 1 -,U 2 \V 2 )+r l ), ^ 

from the joint typicality lemma by considering ( 1 /j , (7|) as one 
variable and the fact that Up ~ 111=1 P( u 2 ,iK’ 2 ,i)- Therefore, 
Pr(si = Si, sj 7 ^ si,s 2 = s 2 , s 2 = s 2 ) tends to zero as 
n —► 00 if 

i?t / 2 > 7(C7 2 ; P 2 IP 2 ) - I(V U Uu U 2 \V 2 ) 

= I{U 2 ;Y 2 \V 2 ,U 1 ), (75) 

where the last equality is due to V\ — U\ — — U 2 . Similarly, 

Pr(si = Si, si 7 ^ si,s 2 = s 2 ,s 2 = s 2 ) in ( |63j ) vanishes as 
n —> 00 if 

Ru, >I{UuY x \VuU 2 ). (76) 

Gathering (|65[i-(f76|, probability of error event Eg in ( |55j ) 

vanishes as n —> 00 if, for j,j' = 1,2 and j 7 ^ j' , the rates 
Rj = Ry j + Ruj satisfy 

Ri>I(Ui',yi\U 2 ), (77) 

R 2 >I{U 2 -Y 2 \U 1 ), (78) 

Ri + R 2 > I(U\, U 2 ; Y\,Y 2 ), (79) 

R, U3 >I(U f Y ] \V ] ,U :i ,), (80) 

Ru,+Ru 2 > I{UuU 2 -,YuY 2 \VuV 2 ), (81) 

Rj + Ru :i , > I(U X , U 2 \Y\,Y 2 \V y ). (82) 

Finally, Pr(E) in ( [54] ) tends to zero as n —> oc provided 
that {77])-<[82]l are satisfied. 

Equivocation rates : The equivocation rate with respect to 
Agent 1, averaged over all codebooks C n , is written as 

H(X n \f 1 (Y 1 n ),E n ,C n ) = H(X n \b 1 ,w 1 ,E n ,C n ) 

= H(X n ,Y 1 n \b 1 ,w 1 ,E n ,C n ) - H(Yp\buWuE n ,X n ,C n ) 
= H(Yp\h,wuE n ,C n ) + H(X n \b 1 ,wi,E n , Yp , C n ) 

-H(Yp\b 1 ,w 1 ,E n ,X n ,C n ) 

= H {Yp\b 1 ,w 1 ,E n ,C n ) + H(X n | E n , Yp) 

-H{Yp\buWuX n ,C n ) (83) 

where the last equality holds since fi(Yp) = (bi,w±) is a 

deterministic function, Yp~ ( 61 , w\, X n ,C n )—E n is aMarkov 
chain, and (X n ,Yp, E n ) are independent of the codebook. 
The first term in ( [83] ) is bounded as 

H(Yp\h,wi,E n ,C n ) 

= H(Yp\bi, E n ,C n ) - I(YP;w 1 \b 1 ,E n ,C n ) 

> H(Yp\bi,E n ,C n ) - H(w 1 \b 1 ,E n ,C n ) 

> H(Yp\s 1 ,E n ,C n )-H(w 1 \C n ) 

= H(Yp,E n \s\,C n ) - H{E n \s\,C n ) - H( Wl \C n ) 

( = } H(Yp,E n ) - H(si\C n ) + H(s\\Yp , E n ,C n ) 

- H(E n \ Sl ,C„) - H( Wl \C n ) 

> H(Yp , E n ) - H(si\C n ) - H(E n \ Sl ,C n ) - H( Wl \C n ) 

> nH(Y\ , E) - nI{VuYp) - ne 1 - H(E n \ Sl ,C n ) 

— nRjj 1 — nc 4 
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> n[H(Y u E) - /(Vi;Fi) - H(E\V 1 ) - Ru, - e' 

- £l ~ £ 4 ] 

( g ) 

> n[H(Y 1 \E) - /(Vi; Yi\E) - R Vl - e' - e 4 - e 4 ] 

= n[JT(yi|Vi£?) - - e' - ei - e 4 ], (84) 

where 

(a) follows as the bin index ioi is a deterministic function of 

F”; 

(b) follows since the bin index 6 4 is a deterministic function of 
the codeword index s 4 , and conditioning reduces entropy; 

(c) follows since ( Y™,E n ) are independent of the codebook; 

(d) follows since the codeword index s 4 is a deterministic 
function of Y{ 1 \ 

(e) follows since random variables Y\ r and E, are i.i.d., 
si and w\ are random variables over sets of size 
2 «(r(Vi;Yi)+ei) anc j 2 n (Ru 1 +ti), respectively; 

(f) follows from ( [52] ) in Lemma [TJ 

(g) follows from the Markov chain Vi — F 4 — E. 

The last term in ( [83] ) can be bounded as 

H(Y 1 n \b 1 ,w 1 ,X n ,C n ) ( = } H(Y 1 n \b 1 ,w 1 ,U^,X n ,C n ) 

= H(Y 1 n \b 1 ,w 1 ,U?,U?,X n ,C n ) 

+ I(U?-,Y 1 n \bi,w 1 ,U2,X n ,C n ) 

< H(Y?\U?,X n ,C n ) + H(U^\b 1 ,w 1 ,U^,X n ,C n ) 

<n[H(Y 1 \U 1 ,X) + e], (85) 

where 

(a) follows from the Markov chain F"— (b\,w\, X n ,C n )—U 2 \ 

(b) follows since conditioning reduces entropy, the bin index 
W\ is a deterministic function of [/", and the Markov 
chain U\ l - (X rl ,C n ) - U%\ 

(c) follows from Fano’s inequality (i.e., in the decoding 
scheme having U%, 61 , and w \ , the CEO decodes [/{' 
with high probability). 

Substituting ( f84] > and ( [85] ) in ( [83] ), we obtain 

H(X n \f 1 (Y 1 n ),E n ,C n ) 

> n[H(Y 1 \V 1 ,E)-R Ul +H(X\Y 1 ,E)-H(Y 1 \U 1 ,X) 

- £ - £ - £ -\ - e 4 \. (86) 

Therefore, 

Ai < H(Y 1 \V 1 ,E) - R Vl + H(X\Yi,E) - H(Y 1 \U x ,X) 

(87) 

is achievable. Based on ( [80] ), using Fourier-Motzkin elimina¬ 
tion yields 

At < H{Yi\V\,E) - I(U X 'Y rlVr, t/ 2 ) + H(X\Y U E) 
-H(Yi\lh,X) 

= H(Y 1 \V 1 ,E)-I(U 1 -,Y 1 \V 1 ,U 2 ) + H(X\Y 1 ,E,V 1 ) 

— H(Yi\Ui, V\, X) 

= H(Y 1 ,X\V 1 ,E) - I(U 1 -Y 1 \V 1 ,U 2 ) 

— H(Yi\Ui,Vi, X) 

= H(X\Vi, E) + H(Y 1 \V 1 ,X,E) - I(U X ',Y i|Vi 5 1/ 2 ) 


-H{Yi\Ui,V u X) 

= H(X\Vi,E) - I(!h-,Yi\V u U 2 ) + I(Uy, Yi| Vi, X), 

(88) 

where 

(a) follows from the Markov chains V\ — (U\. X) — Y t and 
Vi-(Y 1 ,E)-X; 

(b) follows from the Markov chain Y\ — (V±,X) — E. 
Similarly for the second link, we obtain 

A 2 < H(X\V 2 , E) - I(U 2 ; Y 2 \V 2 , t/i) + I(U 2 ;Y 2 \V 2 , X). 

(89) 

For the sum of the equivocation rates, following the proof 
of the equivocation rate, we have 

A 1 +A 2 <H(X\V 1 ,E)+H(X\V 2 ,E)-R Ul - r U 2 
+ I(Ui;Yi\Vi,X) + 1 ( 1 / 2 ', F 2 \V 2 ,X) 

< H(X\V 1 ,E) + H(X\V 2 ,E) 

— I(U\, U 2 ',Yi,Y 2 \Vi,V 2 ) 

+ I(U 1 -,Y 1 \V 1 ,X) + I(Ur,Y 2 \V 2 ,X), (90) 

where we substitute ( [8T| in the second inequality. In order to 
prove we have 

Ai < H(X\V U E) - R Lh +I (UyY^X) 

< H(X\V\,E) — I(Ui,U 2 ; Fi, Y 2 \V X ) + R 2 
+ I(U 1 -Y 1 \V 1 ,X) 

= H(X\V 1 ,E) - I(UrMVi) - I (Ur, Y 2 \V x ,U x ) 

+ R 2 + I(U 1 ',Y 1 \V 1 ,X) 

= H(X\V 1 ,E) - I(Ui',Y\\V\) - I(U 2 \ Y 2 \U\) + R 2 
+ I(U 1 -,Y 1 \V 1 ,X), (91) 


where 

(a) follows from ( [82] ); 

(b) follows since U-\ — F 4 — Y 2 - U 2 is a Markov chain; 

(c) follows since V\ - U\ - U 2 forms a Markov chain. 

Inequality ([9]) can be proved similarly. 

Distortion at the CEO'. The distortion at the CEO is written 

as 


E 


d(x n , g(fi(Y™), / 2 (F 2 ")))] < Pr(E)d max 

+ Pr(E c )E[d(x", X«( Sl) s[),u%(s 2 , 4))) 

1 71 

+ - ^E d^Xi, X (tii,i(si, 4)i u 2 ,i( s 2, 4))) 

= 1 

d(X,X(U u U 2 )) 


(«) 1 
< £n~\ - 

n 


(b) 

< £ 


E 


E 

(92) 


where 

(a) follows since the probability of an error event becomes 
small as n —> 00 if the rate constraints are satisfied. 
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(b) follows since for every jointly typical sequences 

(x n , m", u 2 ), we have 


d(x n ,X(ui,U2)) = ^ y]d(xj,X(u 1> i,u 2 ,i)) 

‘ 1 

d(x,X(u i, 

AT (a;, izi, w 2 |^ n , u ™, E?) 


2=1 


E 

( 01,221 ,222)EA’ XWi X^2 


= E 


d(X,X(U u U 2 )) 
1 

H— 
n 


d(x,X(ui,u 2 )) 


E 

(x,ni ,222)(=A' XU 1 XU 2 
(^ — N(x, Ui,U 2 \x n , m", M 2 ) ~ «1> u 2 


< E 


d(X, X{U\,U 2 )) +d max \X\\U 1 \\U 2 \e l n , (93) 


with e' n —> 0 as n —> 00 from properties of jointly (strong) typ¬ 
ical sequences. Therefore, to satisfy the distortion constraint 
at the CEO, it is sufficient to have 


E 


d(X,X{U u U 2 )) 


< D. 


This concludes the proof of Theorem [T] 


(94) 


Appendix B 

Proof of PropositionQ] Cardinafities in the Inner 
Bound 

To bound the cardinality of alphabets of auxiliary random 
variables V\ and U\ in Theorem |TJ we rewrite the equations 
of the inner bound using straightforward derivations as 


Ri >^(Xi|U 2 )- U 2 ), 

(95) 

R2>I(U 2 ;Y 2 \U 1 ), 

(96) 

Ri+R2> H(Y j) - 77(141(7!) + I(U 2 -,Y 2 \Ui), 

(97) 

At < [l(X;U 2 \Vi) — I(X; E\Vi) + H(X\Ui, U 2 )] + , 


(98) 

A 2 < [H(X\V 2 ,E)-H(X\V 2 ,U 1 )+H(X\U 1 

,u 2 )] + , 


(99) 

Ai+A 2 < [l(X; V 2 I Vj) — I(X-,E\Vi) 


+ H(X\V 2 ,E) + H(X\U u U 2 )] + , 

(100) 

A 1 -R 2 < [l{X\ U 2 \V\) — I{X\E\V\) 


— I(U 2 -,Y 2 \Vi) + H(X\Ui,U 2 )] + , 

(101) 

A 2 — R± < [H(X\V 2 ,E) - H(Yi\U2) 


+ H{Y 1 \U 1 ,U 2 )-I(U 2 ;X\V 2 )] + , 

(102) 

D > E\d(X,X(U u U 2 )) . 

(103) 


Using standard arguments based on the Fenchel-Eggleston- 
Caratheodory theorem and the support lemma ]39j 
Appendix C], it can be shown that Vi should have |34| — 1 
letters to preserve the probability distribution p{y\), and ten 
more to preserve H{Yx\Ui,U 2 ), I(U 2l Y 2 \Ui), H(Y l \U 1 ), 
I{X-U 2 |Vi), I{X-E\Vx), H{X\U u U 2 ), H(X\V 2 ,lh), 
I(X\V 2 \V\), I(U 2 ; Y 2 \V\), and the distortion constraint. 


Furthermore, for each vi £ Vi, there exists a set U\ with 
|34| — 1 letters to preserve the distribution p{yi), plus six 
more to preserve H(Y 1 \U 1 ,U 2 ), I(U 2 -,Y 2 \U{), H(Y 1 \U 1 ), 
H(X\Ui,U 2 ), H(X\V 2 , Ui), and the distortion constraint. 

Therefore, it suffices to have |Vi| < |34 + 9 and \U\\ < 
(1341 + 9)(|3 ? i| + 5). The same holds for the cardinalities of 
the sets V 2 and U 2 . ■ 


Appendix C 

Proof of Theorem[ 2 ] The Outer Bound 
We denote the message transmitted by the jth agent as Jj = 
fj ( YJ 1 ), and also define auxiliary random variables 

Vj ti := {Jj 1 X i ~ 1 ), (104) 

Uj,i ■= 

= (J j ,X i ~ 1 ,Yr\Yt 1 ), (105) 

for j £ { 1, 2} and i £ {1,..., n}. Then, we have the following 
chain of inequalities for Agent l’s rate: 

n(R i + e) > H(Ji) 

I(Ji-,J 2 ,X n ,YT,Y?) 

> ,Y™,Y£\J 2 ) 

n 

= Y I{Ji\Xi, Y lti , Y 2>i \J 2 , X*- 1 , Yt^Yt 1 ) 

i=1 

( d ) EE 

> E J ( J i;Xl.tl J 2 ,x^^-^Yt 1 ) 

2 = 1 

n 

= E 7 ( J 1 . ^ i_1 . Yt 1 , Yt 1 ; il.il J 2 , X*- 1 , Yt 1 , Yt 1 ) 
2=1 
n 

( = } E ^ 14 ; Xi.il u 2,i), (106) 

i=1 

where 

(a) follows as Ji = fi(Y™); 

(b) follows from the non-negativity of mutual information; 

(c) follows from the chain rule of conditional mutual infor¬ 
mation; 

(d) follows from the non-negativity of mutual information; 

(e) follows from the definition of auxiliary random variables 
in (fr04|-(fl05|. 

The equivocation with respect to Agent 1 is bounded as 

n{A 1 ^e)<H{X n \J 1 ,E n ) 

= H{X n \J 1 ) - I{X n -E n \J 1 ) 

= H(X n \J 1 ) - I(X n -E n ) + 7(Ji; E n ) 

n 

= Y [H&ilJuX*- 1 ) -I{Xi-,Ei) +I{J u E l ~ 1 ;E l ) 

2=1 

( C ) U r 

< E [HiXilJ^X*- 1 ) + IiJuX^^^^Ei) 

2=1 

-IiXuEi) 

n 

= E [niXiiJ^x*- 1 ) -iiXi-E^x*- 1 ) 

2=1 
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n 

Y [H(Xi\V lti ) - IiX^EilV^) 
2 — 1 

n 

Y \H(Xi\Ei)- IiXi-V^Ei) 


2—1 


(107) 


where 

(a) follows since Jr — X n — E n is a Markov chain; 

(b) follows since Xi and Ei are memoryless, and the chain 
rule of conditional entropy; 

(c) follows the non-negativity of mutual information; 

(d) follows from the Markov chain E i —X~(Ji,X l ~ 1 )—E l ~ 1 ; 

(e) follows from the definition of auxiliary random variable 
in ( fl04l >. 

To prove ( | 1 6| >, we have 

n(Ai — e) < H(X n \J\, E n ) 

= H(X n \Ji, J 2 , E n ) + I(X n ; J 2 | Jr, E n ) 

(a) 


< [ H (Xi\Ji,J2,X i -\E n )] +H(J 2 ) 

2=1 

(&) " r . i -I 

< 55 {HiXilJ^J^X'-^E^ +H{J 2 ) 

2=1 

n 

Y [ H (Xi\V^V^Ei)] +H(J 2 ) 


2=1 


Th 

< Y [mXilV^Ei) - HXiM^V^Ei) 


2 = 1 


2=1 


- i{x'~\ Y{-\ KT 1 ; Xi, Y hi , y 2ii ) 


= 55 /(.A, J 2 ,x i-1 , f/- 1 , kt 1 ; jq, y M , y 2 ,i) 


2=1 


(d) 


> 55 J ( J i> J 2 , x 1 - 1 ,r/- 1 ,kt 1 ; n,i,y 2 ,i) 


2 = 1 


(e) 


= 55 / (^M’^; F M’ y 2.*)’ 


(a) follows as Ji = /i(y L n ) and J 2 = / 2 (y 2 n ); 

(b) follows from the chain rule of mutual information; 

(c) follows since X, n Y^,, Y 2 l are memoryless; 

(d) follows from the non-negativity of mutual information; 

(e) follows from definition of auxiliary random variables 
( fl05l ). 

Next, the distortion bound at the CEO is written as 


D + e> E 
1 

> - 
n 


d(X n , g(J\, J 2 )) 


n 

Y^[d(x i ,X i (U hi ,U 2 , i )') 


( 110 ) 


+ nR 2 + ne, 
(108) 

where 

(a) follows from the non-negativity of conditional mutual 
information and the fact that conditioning reduces entropy; 

(b) follows since conditioning reduces entropy: 

(c) follows from definition of auxiliary random variables 

( fl04l ). 

Since the setup is symmetric, bounds on Agent 2’s transmis¬ 
sion and equivocation rates in ( fT2) , ( fT5| , and ( fl7f > are derived 
similar to those of Agent 1. The sum rate is bounded as 

n(R-\ + R 2 + 2e) > i/(Jr, J 2 ) 

= I(J 1 ,J 2 -,X n ,Y 1 n ,Y 2 n ) 

n 

= 55 /(Jr, J 2 ; Xi, Y hi , y 2j! ; |A—\ yr 1 ) 

2=1 

n 

= 55 /(Jr, J 2 , X l ~\Yt\Yt 1 ; Xi, y M , y 2 ,i) 


where Xi(Uij,U 2 ^) := gi{Ji, J 2 ) is the zth element of the 
CEO’s decoded sequence. 

We define a time sharing random variable Q independent 
from all other random variables and uniformly distributed over 
the set {1,... ,n}. We further define X = Xq, Yj = Yj^q, 
E = Eg, Uj = (Q, U jtQ ), and Vj = (Q, V jlQ ) for j e {1,2}. 
Also let X(lh,U 2 ) = X(Q,U hQ ,U 2 ,g) = X Q {Jh, Q ,U 2jQ ). 
We have the Markov chains Vj — Uj — Yj — ( X,E,Yj> ) for 
j, j' £ {1,2} and j ^ j'. Now, we can write the rates and 
equivocation bounds for Agent 1 (and similarly for Agent 2) 
as follows: 

1 ” 

Ri+e> -55/(// m ;Y i, i\U 2 ,i) 
n z ' 

i=l 
1 ” 

= — 55/(C/i,q; Yi,q\U 2,Q, Q = i) 


i—1 


A, - 


= I(U 1 ;Y 1 \U 2 ), 

1 
n 


(111) 


1 " 

e < - 55 \ H ( X i\ E i) - HX* Vi,i\Ei) 


2=1 


'll 

55 [ h ( x q\Eq,Q = i) 


2=1 

-I(X Q -,V hQ \E Q ,Q = i) 
= H(X\E) - IiX-V^E), 

Ar — R 2 — 2e 
1 


( 112 ) 


< ^55 [H(Xi\V hi ,Ei) ~ I(Xi-V 2t i\V lti ,Ei) 

i=l 
n 

55 [H(X Q \V hQ ,E Q ,Q = i) 


1 

n 


i—l 


(109) 


- I{X Q -V 2<Q \V hQ , Eg,Q = i) 

= H(X\V 1 ,E)-I(X-,V 2 \V 1 ,E). (113) 

The sum rate and the distortion bounds can also be written as 
1 n 

Ri + R'2 + 2e > — 55 1(U lti , U 2 f, Yu, Y 2 i) 

2=1 

1 n 

= ~ YJ{Ui,q, U2,q-,Y 1iQ ,Y 2i q\Q = i) 

1 i= 1 

= /(c/r,f/ 2 ; yi,y 2 ), 


where 


(114) 
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D + e > 


_ 1 
n 

= E 


= E 


i n 

-J2 E 

U i =1 
n 

y~l E d[xQ,XQ{Ui i Q 1 U2,o) s j 

i—1 

d(x Q ,X Q (U 1 >Q ,U 2:Q )) 
d(x,X{U u U 2 )) 


Q = i 


(115) 


Letting e and e n tend to zero, inequalities of the Theorem [2] 
follow. ■ 


Appendix D 

Proof of Proposition [2] Cardinalities in the Outer 

Bound 


For the equivocation rates, we have 

n(Ai -e) < H{X n \J ± ) 

= H(X n ,J 2 \J 1 )-H(J 2 \J 1 X n ) 

= H(J 2 |Ji) - + H{X n \J x J 2 ) 

< H(J 2 ) — H{J 2 \X n ) + ne n 
= I(J 2 -,X n )+ne n 

n 

= Y / I{J 2 X i ~ 1 -X i )+ne n 

i— 1 
n 

= Y , I ( U *> i > Xi ) + ne n , 024) 

j=i 

where (a) follows from Fano’s inequality, the Markov chain 
Ji — X — J 2 , and the fact that conditioning reduces entropy. 
We also have 


To bound the cardinality of the sets Vi and U\ in Theorem [2] 
we rewrite the equations of the outer bound as 



Ri 

> 

H(Yi\U 2 ) — H(Y\\U\,U 2 ), 

(116) 


R‘2 

> 

I{U 2] Y 2 \UJ, 


(117) 

Ri 

+ R 2 

> 

H{Y 1 )-H{Y l \U 1 ) 

F I{U 2 ;Y 2 \Ui), 

(118) 


A 1 

< 

H{X\V 1 )-I{X-,E\V 1 ), 

(119) 


a 2 

< 

H(X\V 2 )-I{X;E\V 2 ), 

(120) 

A 1 

-Ri 

< 

H{X\V 1 ) - I(X- E\Vx) - I{X\ y 2 |I4) 






+ I{E-V 2 \V 1 ), 


(121) 

a 2 

-Ri 

< 

£ 

1 —1 

t? 

V^-IiX-E^) 






+ I{E-V 2 \V 1 ), 


(122) 


D 

> 

E 

d{X,X{UpU 2 )) 


(123) 


Similar to the approach in Appendix [B] using standard ar¬ 
guments based on the Fenchel-Eggleston-Caratheodory the¬ 
orem and the support lemma [39| Appendix C], it can 
be shown that Vi should have |34] — 1 letters to pre¬ 
serve the probability distribution p{yi), and eight more to 
preserve H^UpUi), I(U 2 ;Y 2 |t/r), 7T(Fi|17i), H{X\Vi), 
I(X; E\Vi), I{X ; V^Vi), I(E; V 2 \Vi), and the distortion con¬ 
straint. Furthermore, for each v-\ G Vi, there exists a set U\ 
with 1 3 ^ 1 1 — 1 letters to preserve the distribution p{yi), plus four 
more to preserve H(Y\\U\, U 2 ), I(U 2 ; Y 2 \U\), H(Y\\U\), and 
the distortion constraint. 

Therefore, it suffices to have |Vi| < 3^i + 7 and \U\\ < 
(13^11 + 7)(|3i| + 3). The same holds for the cardinalities of 
the sets V 2 and U 2 . ■ 


Appendix E 

Converse Proof of Corollary[4] 

Let J\ = fi(X n ) and J 2 = f 2 (Y 2 ) denote the messages 
transmitted by Agent 1 and Agent 2, respectively, and also 
define the auxiliary random variable U 2l = (J 2 , X^ 1 ), for 
i G {I,---, n}. 


n(Aa-c) < H(X n \J 2 ) 

n 

= Y J H(X t \J 2 X l ~ 1 ) 

1=1 

n 

= Y / H(X i \U 2>i ). (125) 

i=1 

Similar to the proof of Theorem [2] by using an independent 
random variable Q , inequalities of Corollary [4] follow. ■ 


Appendix F 

Proof of Theorem[3} Quadratic Gaussian Case 
without Side Information at Eve 


A. Achievability 

The achievability is proved by applying the results of 
Theorem |T] to the Gaussian case. Although Theorem [T] is 
proved for finite alphabet sources with bounded distortion 
measure, its results can be applied to the Gaussian sources 
with quadratic distortion measure as shown e.g., in |40|. In 
addition similar results are proved for the Gaussian CEO 
problem (without secrecy constraints) in [ 131 and GB¬ 
it follows from Theorem Q] that with no side information at 
Eve the following region is achievable: 


ri^hu^ym, 

R 2 >I(U 2 -,Y 2 \U 1 ), 

Ri + R 2 > IiUuUxY^Yi), 

Ai < h(X) — I {UpY i\U 2 ) + I(Ui;Yi\X), 
A 2 < h(X) - I {Up Y 2 \Ui) + I(U 2 ;Y 2 \X), 
A 1 +A 2 < 2h{X) I(U \, U 2 ; Y\,Y 2 ) 

+ I(U 1 -,Y 1 \X) + I(U 2 -,Y 2 \X), 

At - R 2 < h(X) - I{U 2 ; y 2 |I7r) - I {Up, Y x ) 

+ I(U 1 -,Y 1 \X), 

A 2 - f?i < h(X) - I{Up Yi\U 2 ) - I{U 2 -Y 2 ) 

+ I(U 2 -Y 2 \X), 


D> E 


d(X, X{Up U 2 )) 


(126) 

(127) 

(128) 

(129) 

(130) 

(131) 

(132) 

(133) 

(134) 
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where h (■) is the differential entropy. This region is obtained 
by setting V\, V 2 , and E to be constant in |5 ]m|9Ji. 

Let U\ and U 2 be auxiliary random variables jointly dis¬ 
tributed with the source A' and the respective agent’s obser¬ 
vation Vj and Y-> such that 

Uj =Yj+ Zj, j £ {1,2} (135) 

where Zj ~ J\f(0, a 2 z .) for j £ { 1, 2} is independent of X, Y \, 
and Y 2 . Given U\ and U 2 , we choose the estimator function 
X(U\. U 2 ) to be the minimum mean square error (MMSE) 
estimator. For a fixed target distortion D > 0, the auxiliary 
random variables are adjusted so that the distortion constraint 
(]134|) is satisfied. Therefore, we have 


Now, for the rate of Agent 1, from (| 126|> we have 


Ri > J(£/i;Yi|tf 2 ) 

== I(Ui; X, Yi\U 2 ) 

( 6 ) 


= I(Ui-,X\U 2 ) + I(lh MIX) 


— h(X\U 2 ) — h(X\Ui, U 2 ) + ri 


(d) 1, / l / l 

= - 2 log (,^lst + 


1 2~ 2r2 


’No 


- -log(27reD) + n 

L 1 1 , / 1 

= o lo § 73 - o lo § zr + 


D 


1 - T 


X 


’n 2 


+ r 1 , 


(143) 


D = E 


d(X,X(U 1 ,U 2 )) 


1 Q2fe(-X-|i7i,17 2 ) 
27re 


E 


(X -X(U lt u 2 )Y 


(136) 


where ( | 1 36[ > follows from [37] Theorem 8.6.6]. We then 
calculate 


where 

(a) follows as U\ — kj — (X, U 2 ) forms a Markov chain; 

(b) follows as (Ui,Yi) — X — U 2 is also a Markov chain; 

(c) follows from the definition of n in ( |139| ); 

(d) follows from ( | 136| > and ( | 141 1 >. 

The rate of Agent 2 in ( ]127| ) is written in a similar way. For 
the sum rate, we have 


h(X\U u U 2 ) = h(X, lh, U 2 ) - h(U u U 2 ) 




det K X u 1 u 2 \ 
det Ku ± u 2 ) 


(137) 


where the covariance matrix K \ u, u 2 is 


K xu 2 



a x 

Vx 


Vx + °N ! + ^Ix 

* 2 X 

0 % 

0 % 

°x + °n 2 


Ri+R2>I(U 1 ,U 2 -,Y 1 ,Y 2 ) 

^ I{U 1 ,U2-,X,Y 1 ,Y 2 ) 

= I(U 1 ,U 2 ; X) + I(U\,U 2 \ Y u Y 2 \X) 

= h(X) - h(X\U lt U 2 ) + HUuY,\X) + I(U 2 -,Y 2 \X) 

- \ log(27recri-) - ^ log(27rei>) + n + r 2 
= 2 l0g Z)-2 l0g ^ +ri+r2 ’ (144) 


Substituting ( | 137[ > in ( | 1 36[ i, we obtain 


1 _ 1 1 
D a 2 x + a 2 Ni + a 2 Zi 


1 


’n 2 


+ a 


2 • 

z 2 


(138) 


where (a)-(c) are due to the long Markov chain U\ — Y\ — 
X — Y 2 — U 2 and (d) from ( | 1 36] and ( fl39| . 

Using ( 139| ) and the bound on R.\ (and R 2 ), the equivocation 
rate in \\29\ (and similarly (|130|>) can be written as 


Next, for j £ {1, 2}, we define 


r 3 := I{U r ,Yj\X) 

= I(Uj',Yj) — I(Uj;X) 


J Ni 


1. a 7 
= 2 bg - CT | 


(139) 

(140) 

(141) 


where the second equality is due to the Markov chain Uj—Yj — 
X. For any r :i > 0 with j £ {1, 2}, there exists corresponding 
o 2 z ., and thus, auxiliary random variable Uj. Then, using 
(| 141 [>, we can rewrite (] 138 [) as 


1 _ 1 1 — 2 -2ri 
D (7 X + a N 2 


1 - 2~ 2l ~ 2 


a 


2 

n 2 


( 142 ) 


Ax < h{X)-I{U l -Y 1 \U 2 ) + I{U 1 -Y l \X) 

1, 2x1, 1 1, / 1 1 - 2" 2r2 \ 

= 2 log(2,r “ 7 * ) - 2 log 5 + 2 log (, 5 + ) ■ 

2 (145) 

For sum of the equivocation rates, we have 

Ax + A 2 < 2 h(X) - I(U 1 ,U 2 -,Y 1 ,Y 2 ) + J(Ux;Fi|X) 

+ I(U 2] Y 2 \X) 

= log(2.ea|)-ilog-i + ilog^ 

= log(27re) - ^log-^ - ^log^-. (146) 

Finally, the bound in (| 1 32|> (and similarly (|133|)) can be 
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rewritten as 


Based on Lemma [2j (|150|), and (| 149|>, we obtain 


Ai — R 2 

< h(X) - I{U 2 -Y 2 |J7i) - I(Ui]Yi) + I{U x] Y x \X) 

= h(X) - I{U X ,U 2 ; Y x ,Y 2 ) + I(Ui;Yi\X) 

= \og{2nea%)~ b log + \ log Jr - r i - r 2 + ri 

= log(27re) - -log — - r 2 , (147) 

where (a) is due to the Markov chains U\ — Y x — Y 2 and 
Y x — Y 2 — U 2 . U 


B. Converse 


Denote the messages transmitted by the agents as J x = 
/-, (Y{ 1 ) and J 2 = f 2 (Y.j l ), and define 

rj := ±I(J j] Y?\X n ). (148) 

Let J be a subset of {1,2} and J r its complement. We have 


J2n(Rj+e)>H(Jj) 

jej 

> ) 


= I(Jj-,Y$\Jj.) 

= I{Jj-,X n ,Y$\Jjc) 

= I(Jj ; X n \Jjc) + I(Jj\ YJ\X n , Jj ,) 

= J(Jy, Jjc-X n ) - I(Jjc-X n ) + I(Jj\Y?\X n ) 


(c) 

> 


I(X n -X n )-I(Jj.-X n ) 


n + 


' £ nr ’ 
j£J 


Ji 


(149) 


where 

(a) follows from the Markov chain Jj — (Yj,Jjo) — X n \ 

(b) follows from the Markov chain (Jj^YJ 1 ) — X n — Jy for 

j 7^ /; 

(c) follows from the data processing inequality and the defi¬ 
nition of i-j in ( | 148| >. 

The first term in the right-hand side of ( |149[ > is bounded as 

I{X n -X n ) = h{X n ) - h{X n \X n ) 

= h{X n ) - h(X n - X n \X n ) 

> h{X n ) - h{X n - X n ) 

> | log(27recr^) - | log(27reJ9) 

= (150) 


We use the following lemma to bound the second term in the 
right-hand side of ( | 149| >. The proof of the lemma is given in 
O and (14). 

Lemma 2: Define rj := 2/( Jj', Y J n \X n ) for j £ {1,2}. 
Then, for J C {1, 2}, 


-I(Jj-X n )<\ log 
n Z 


'x 



+ £ 

o£j 


1 - 2~ 2rj \ 

a k ) 


v- 1,11, 


+ £ r r 

jeJ 



+ £ 


1 - 2 - 


J N 4 


(151) 


Now, substituting J with sets {1}, {2}, {1,2}, and 0 leads 
to the bounds ([30])—([32]) and ( [38] ) of Theorem [3] respectively. 
The bounds on the equivocation rates are obtained as 


£ n( A,- - e) < £ K xn \ J j) 


3&J 




= E h ( xn ) - H ( J j) + H ( J j\ xn ) 


jej 


(a) 




jej 


(b) 


[O) _ r 

< E [K xn ) + I{ J Y Y ?\ xn )\ -H(Jj) 


jej 


( c ) n , , . n , 1 

< E 3 lo s( 27recr x) “ 9 l°g 






2 D 
1 - 2~ 2rj 


Gx jej a 


(152) 


N, 


where 


(a) follows since Jj = 

(b) follows since H(Jj) < YljeJ 

(c) follows from the definition of r 3 in ( |148| ) and the same 
techniques used to bound H{Jj) in ( |149| >. 

Setting J to {1}, {2}, and {1,2} leads to the bounds ([33]- 
( |35] , respectively. 

For the bound in ([36], consider 


n(Ar -e) < h{X n \J x ) 

= h(X n \J 1 ,J 2 ) + I(X n -J 2 \J 1 ) 

= h(X n ) - I(X n ; J x , J 2 ) + H(J 2 \J x ) - H{J 2 \J u X n ) 

< h(X n ) - I{X n -X n ) + H(J 2 ) - H(J 2 \X n ) 

< h{X n ) - I{X n ; X n ) + nR 2 - I(J 2 \Y?\X n ) 

(c) 

< - log(27recr|-) - 9 log x + nR 2 - nr 2 , (153) 

where 

(a) follows since J x — 3f” — J 2 and that conditioning reduces 
entropy; 

(b) follows since J 2 = f 2 (YJ 1 '); 

(c) follows from ( |150] and the definition of r 3 in ( |148] , 
Therefore, 

Ai - i? 2 < ^ log(27re) - ^ log ^ - r 2 . 

The bound in ([37] can be proved similarly. 


(154) 
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Appendix G 

Proof of Theorem}!} Quadratic Gaussian Case 
with Side Information at Eve 


The proof of Theorem [4] is based on a similar approach 
for the achievability proof of Theorem [3] described in Ap¬ 
pendix F-A However, in this case due to the side information 
at Eve, setting the auxiliary random variable V :j to a constant 
value in Theorem |T] does not always maximize the equivoca¬ 
tion rates. We rewrite the bounds on the equivocation rates in 
Theorem Q] as follows: 


A! < h{X\E) - + KJJvWX) 

+ I(V 1 -,E)-I(V 1 -,U 2 ), (155) 

A 2 < h(X\E) - IiU^YM + I(U 2 -,Y 2 \X) 

+ I(V 2 ] E) — I(V 2 ; Ui), (156) 

A: + A 2 < 2h(X\E) - KjJuUxY^Yz) 

+ I(U\;Yi\X) + I(U 2 -,Y 2 \X) 

+ I(V 1 ;E) + I(V 2 ,E)-I(V 1 ;V 2 ), (157) 

A, -R 2 < h(X\E) — I(U 2 -,Y 2 \U\) — I(U\-,Y\) 

+ I(U 1 ;Y 1 \X) + I(V 1 -,E), (158) 

A 2 - i?! < h(X\E) - HlhMlUi) - I{U 2 -,Y 2 ) 

+ I(U 2 ;Y 2 \X) + I(V 2 -,E). (159) 

Then, considering the Markov chains Vj — X — ( E,Uj> ) for 
j' j g {1,2} and the fact that the variables X and E as 
well as the auxiliary random variables Z7i and U 2 in ([135} are 
Gaussian, we have four possibilities: 

• E is less noisy than U 2 w.r.t. X => I{V\\ E) > I(V [; U 2 ) 
=> setting V\ = U-\ maximizes ( |155| >: 

• U 2 is less noisy than E w.r.t. X => I(V\\ E) < I{V \; U 2 ) 
=> setting V\ = 0 maximizes ([155}; 

• E is less noisy than U\ w.r.t. X => I(V 2 ; E) > /( V 2 ; U\) 
=> setting V 2 = U 2 maximizes ([156}; 

• U\ is less noisy than E w.r.t. X => I(V 2 ; E) < /( V 2 ; U\) 
=> setting V 2 = 0 maximizes ([156}. 

Therefore, we define 


T\ := maxI{V\\E) — I{V\\ U 2 ) 

Vi 

= max {0, /((7 i; E) — /((7 i; (7 2 )}, (160) 

T 2 := max I(V 2 \E) - /(U 2 ; lh) 

V2 

= max {0 ,I(U 2 ;E)-I(U i; U 2 )}, (161) 

and rewrite <[T55}-([T59} as 

Ar < h(X\E)—I(U 1 ; Yi\U 2 )+I(U 1 -,Yi\X)+Ti, 

(162) 

A 2 < h{X\E)-I{U 2 -,Y 2 \U 1 )+I{U 2 -,Y 2 \X)+T 2 , 

(163) 

A 1 +A 2 <2h(X\E)-I(U 1 ,U 2 -,Y 1 ,Y 2 ) 

+ I(Ui;Yi\X)+I(U 2 ] Y 2 \X) 

+ T 1 +T 2 +I(U 1 -,U 2 )1 r>0 {T 1 +T 2 ), (164) 

Ai R 2 < h(X\E) — I(U 2 ', Y 2 \Ui) — I(Ui;Yi) 

+ I(U 1 -,Y 1 \X)+T 1 +I(U 1] U 2 )1 m>0 (T 1 ), (165) 
A 2 -R 1 < h(X\E) — I(U\\ Yi\U 2 ) — I(U 2 ; Y 2 ) 


+ I(U 2 ;Y 2 \X)+T 2 +I(U 1 -,U 2 )1 r>0 (T 2 ), (166) 


where (■) is the indicator function. Based on the defi¬ 
nitions of the auxiliary random variables in ([135} and rj in 
( fl4T} , with straightforward calculations we obtain 


T\ = max ■ 


T 2 = max 


o.j log 


0, J log 


1 + 


1 + 


"»2 

l_ 2 -2r 2 


j n k 


"«1 

1 — 2 2 r 
2 

<t N 1 

1 —2 -2t ' 


'N e ■ 


— <J 


Ne 


1_ 2 -2t- 2 


+ CT 


Ne ' 


U 2 ) = - log 


cri + 


i 2 - 


"1 


1 - 2 - 


" N 2 

1 — 2 - ^ r 2 


(167) 

(168) 

(169) 


Using ( | 1 62| )— ( fl~69} and the proof given in Appendix |F-A 
inequalities of Theorem [4] follows. ■ 


Acknowledgment 

The authors thank the associate editor and anonymous re¬ 
viewers for their constructive comments. They are also grateful 
to Dr. Kittipong Kittichokechai for his helpful comments on 
an earlier draft of this work. 


References 

[1] F. Naghibi, S. Salimi, and M. Skoglund, “The CEO problem with secrecy 
constraints,” in Proc. IEEE Int. Symp. on Inf. Theory (ISIT), Honolulu, 
HI, Jul. 2014, pp. 756-760. 

[2] Y. Liang, H. V. Poor, and S. Shamai (Shitz), “Information theoretic 
security,” Foundations and Trends in Communications and Information 
Theory , vol. 5, no. 4-5, pp. 355-580, 2008. 

[3] R. Liu and W. Trappe, Eds., Securing Wireless Communications at the 
Physical Layer. Springer, 2010. 

[4] C. E. Shannon, “Communication for secrecy systems,” Bell System 
Technical Journal, vol. 28, no. 4, pp. 656-715, Oct. 1949. 

[5] A. D. Wyner, “The wire-tap channel,” Bell System Technical Journal, 
vol. 54, no. 8, pp. 1335-1387, Jan. 1975. 

[6] I. Csiszar and J. Komer, “Broadcast channels with confidential mes¬ 
sages,” IEEE Trans. Inf. Theory, vol. 24, no. 3, pp. 339-348, May 1978. 

[7] T. Berger and H. Viswanathan, “The CEO problem,” IEEE Trans. Inf. 
Theory, vol. 42, no. 3, pp. 887-902, May 1996. 

[8] S. I. Gel’fand and M. S. Pinsker, “Coding of sources on the basis 
of observations with incomplete information,” Prob. Inf. Transmission, 
vol. 15, no. 2, pp. 45-57, 1979. 

[9] H. Yamamoto and K. Itoh, “Source coding theory for multiterminal 
communication systems with a remote source,” The Trans, of the IECE 
of Japan, vol. 63, no. 10, pp. 700-706, Oct. 1980. 

[10] T. Flynn and R. Gray, “Encoding of correlated observations,” IEEE 
Trans. Inf Theory, vol. 33, no. 6, pp. 773-787, Nov. 1987. 

[11] H. Viswanathan and T. Berger, “The quadratic Gaussian CEO problem,” 
IEEE Trans. Inf. Theory, vol. 43, no. 5, pp. 1549-1559, Sep. 1997. 

[12] Y. Oohama, “The rate-distortion function for the quadratic Gaussian 
CEO problem,” IEEE Trans. Inf. Theory, vol. 44, no. 3, pp. 1057-1070, 
May 1998. 

[13] V. Prabhakaran, D. Tse, and K. Ramchandran, “Rate region of the 
quadratic Gaussian CEO problem,” in Proc. IEEE Int. Symp. on Inf. 
Theory (ISIT), Chicago, IL, Jul. 2004, p. 119. 

[14] Y. Oohama, “Rate-distortion theory for Gaussian multiterminal source 
coding systems with several side informations at the decoder,” IEEE 
Trans. Inf. Theory, vol. 51, no. 7, pp. 2577-2593, Jul. 2005. 

[15] T. A. Courtade and T. Weissman, “Multiterminal source coding under 
logarithmic loss,” IEEE Trans. Inf. Theory, vol. 60, no. 1, pp. 740-761, 
Jan. 2014. 

[16] V. Prabhakaran and K. Ramchandran, “On secure distributed source 
coding,” in Proc. IEEE Inf. Theory Workshop (ITW), Lake Tahoe, CA, 
Sep. 2007, pp. 442-447. 



























17 


[17] D. Slepian and J. Wolf, “Noiseless coding of correlated information 
sources,” IEEE Trans. Inf. Theory, vol. 19, no. 4, pp. 471^-80, Jul. 
1973. 

[18] R. Tandon, S. Ulukus, and K. Ramchandran, “Secure source coding with 
a helper,” IEEE Trans. Inf. Theory, vol. 59, no. 4, pp. 2178-2187, Apr. 
2013. 

[19] D. Giindiiz, E. Erkip, and H. V. Poor, “Secure lossless compression with 
side information,” in Proc. IEEE Inf. Theory Workshop (ITW), Porto, 
Portugal, May 2008, pp. 169-173. 

[20] W. Luh and D. Kundur, “Separate enciphering of correlated messages 
for confidentiality in distributed networks,” in Proc. IEEE Global 
Telecommun. Conf. (GLOBECOM), Washington, DC, Nov. 2007, pp. 
1637-1641. 

[21] D. Giindiiz, E. Erkip, and H. V. Poor, “Lossless compression with 
security constraints,” in Proc. IEEE Int. Symp. on Inf. Theory (ISIT), 
Toronto, Canada, Jul. 2008, pp. 111-115. 

[22] S. Salimi, M. Salmasizadeh, and M. R. Aref, “Generalised secure 
distributed source coding with side information,” IET Communications, 
vol. 4, no. 18, pp. 2262-2272, 2010. 

[23] W. Luh and D. Kundur, “Distributed secret sharing for discrete memo¬ 
ryless networks,” IEEE Trans. Inf. Forensics Security, vol. 3, no. 3, pp. 
1-7, Sep. 2008. 

[24] J. Villard and P. Piantanida, “Secure lossy source coding with side in¬ 
formation at the decoders,” in Proc. Annual Allerton Conf. on Commun., 
Cont., and Comp., Allerton, IL, Sep. 2010, pp. 733-739. 

[25] E. Ekrem and S. Ulukus, “Secure lossy source coding with side 
information,” in Proc. Annual Allerton Conf. on Commun., Cont., and 
Comp., Allerton, IL, Sep. 2011, pp. 1098-1105. 

[26] J. Villard and P. Piantanida, “Secure multiterminal source coding with 
side information at the eavesdropper,” IEEE Trans. Inf. Theory, vol. 59, 
no. 6, pp. 3668-3692, Jun. 2013. 

[27] K. Kittichokechai, Y.-K. Chia, T. Oechtering, M. Skoglund, and 
T. Weissman, “Secure source coding with a public helper,” in Proc. 
IEEE Int. Symp. on Inf. Theory (ISIT), Istanbul, Turkey, Jul. 2013, pp. 
2209-2213. 

[28] Y.-K. Chia and K. Kittichokechai, “On secure source coding with side 
information at the encoder,” in Proc. IEEE Int. Symp. on Inf. Theory 
(ISIT), Istanbul, Turkey, Jul. 2013, pp. 2204-2208. 

[29] R. Tandon, L. Sankar, and H. V. Poor, “Discriminatory lossy source 
coding: Side information privacy,” IEEE Trans. Inf. Theory, vol. 59, 
no. 9, pp. 5665-5677, Sep. 2013. 

[30] H. Yamamoto, “Coding theorems for Shannon’s cipher system with 
correlated source outputs, and common information,” IEEE Trans. Inf. 
Theory, vol. 40, no. 1, pp. 85-95, Jan. 1994. 

[31] -, “Rate-distortion theory for the Shannon cipher system,” IEEE 

Trans. Inf. Theory, vol. 43, no. 3, pp. 827-835, May 1997. 

[32] C. Schieler and P. Cuff, “Rate-distortion theory for secrecy systems,” 
IEEE Trans. Inf. Theory, vol. 60, no. 12, pp. 7584-7605, Dec. 2014. 

[33] F. Naghibi, S. Salimi, R. Thobaben, and M. Skoglund, “The lossless 
CEO problem with security constraints,” in Proc. Int. Symp. on Wireless 
Communication Systems (ISWCS), Ilmenau, Germany, Aug. 2013, pp. 
234-238. 

[34] A. D. Wyner and J. Ziv, “The rate-distortion function for source coding 
with side information at the decoder,” IEEE Trans. Inf. Theory, vol. 22, 
no. 1, pp. 1-10, Jan. 1976. 

[35] T. Berger, “Multiterminal source coding,” in The Information Theory 
Approach to Communications, G. Longo, Ed. Springer-Verlag, 1978, 
pp. 171-231. 

[36] S.-Y. Tung, “Multiterminal source coding,” Ph.D. dissertation, Cornell 
University, NY, 1978. 

[37] T. M. Cover and J. A. Thomas, Elements of Information Theory. John 
Wiley & Sons, 2006. 

[38] I. Csiszar and P. C. Shields, “Information theory and statistics: A 
tutorial,” Foundations and Trends in Communications and Information 
Theory, vol. 1, no. 4, pp. 417-528, 2004. 

[39] A. El Gamal and Y.-H. Kim, Network Information Theory. Cambridge 
University Press, 2011. 

[40] Y. Oohama, “Gaussian multiterminal source coding,” IEEE Trans. Inf. 
Theory, vol. 43, no. 6, pp. 1912-1923, Nov. 1997. 



